In-network Neural Networks

Network devices, such as switches and routers, process data at rates of terabits per second, forwarding billions of network packets per second. Recently, such devices’ switching chips have been enhanced to support new levels of programmability [3]. Leveraging these new capabilities, a switching chip’s packets classification and modification tasks can now be adapted to implement custom functions. For example, researchers have proposed approaches that rethink load balancers [11], key-value stores [7], and consensus protocols [5] operations. In general, there is a trend to offload to the switching chips (parts of) functions typically implemented in commodity servers, thereby achieving new levels of performance and scalability. These solutions often offload some data classification tasks, encoding relevant information, e.g., the key of a key-value store entry [10], in network packets’ headers. Unlike packets’ payload, the header values can be parsed and processed by the switching chips, which perform classification using lookup tables. While providing very high throughput, lookup tables need to be filled with entries that enumerate the set of values used to classify packets, and therefore the table’s size directly correlates to the ability to classify a large number of patterns. Unfortunately, the amount of memory used for the tables is hard to increase, since it is the main cost factor in a network device’s switching chip [3], accounting for more than half of the chip’s silicon resources. In this paper, we explore the feasibility of using an artificial neural network (NN) model as classifier in a switching chip, as a complement to existing lookup tables. A NN can better fit the data at hand, potentially reducing the memory requirements at the cost of extra computation [9]. Here, our work builds on the observation that, while adding memory is expensive, adding circuitry to perform computation is much cheaper. For reference, in a programmable switching chip the entire set of computations is implemented using less then a tenth of the overall chip’s area. To this end, we implement N2Net, a system to run NNs on a switching chip. We provide the following contributions: first, we show that a modern switching chip is already provided with the primitives required to implement the forward pass of quantized models such as binary neural networks, and that performing such computation is feasible at packets processing speeds; second, we provide an approach to efficiently leverage the device parallelism to implement such models; third, we provide a compiler that, given a NN model, automatically generates the switching chip’s configuration that implements it. Our experience shows that current switching chips can run simple NN models, and that with little additions a chip design could easily support more complex models, thereby addressing a potentially larger set of applications.